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(54) INTERFACE DEVICE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To improve operability by easily 
operating equipment without requiring any input device such 
as keyboard or mouse by displaying the feature of shape of 
a hand recognized by a recognizing means on a screen as a 
special shape. 

SOLUTION: When a user projects one finger while facing 
equipment provided with an interface device, the icon of 
numeral T is displayed on a display 2 and the television of 
1st menu 201 is displayed while being emphasized. At such 
a time, it is also available to output a sound or a voice from 
the display device 2 corresponding to the emphasized 
indication. In this case, when the user changes the shape of 
the hand to project two fingers, the icon of numeral '2' is 
displayed on the display 2 and the network of 2nd menu 202 
is displayed while being emphasized. By keeping the same 
shape of the hand for a fixed time in such a state, for 
example, the 2nd menu 202 is selected and it can be 
commanded to a host computer 1 to perform the display of 
network terminal. 
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1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim l]An interface device comprising: 

A recognition means to recognize shape of an operator's hand. 

A displaying means displayed on a screen by making into special shape the feature of shape of a 
hand recognized by said recognition means. 

A control means which controls information displayed in said screen by special shape displayed 
on a screen by said displaying means. 

[Claim 2]The interface device according to claim 1 which a recognition means displays on a 
screen by making into special shape shape of a hand that have recognized a motion of said hand 
with shape of a hand, and a displaying means has been recognized by said recognition means, 
and the feature of a motion. 

[Claim 3]The interface device according to claim 1 controlled so that a control means chooses 
information displayed in a screen. 
[Claim 4] An interface device comprising: 
It is an image pick-up part at least. 

shape of an object in a picturized picture — and — or an operation recognition part which 
recognizes a motion. 

shape of an object recognized by said operation recognition part — and — or a frame memory 
which saves a picture which was provided with an indicator which displays a motion and was 
picturized in said image pick-up part. 

Picture difference operation part which provides a reference picture memory which accumulates 
a picture picturized at time before a picture saved in said frame memory as a reference image, 
and extracts a difference between a picture in said frame memory, and a reference image 
accumulated into said reference picture memory in said operation recognition part. 

[Claim 5]The interface device according to claim 4 which provided a reference image updating 
section which updates a reference image accumulated into a reference picture memory in a new 
picture. 

[Claim 6]The interface device according to claim 4 which formed a timer which calculates an 
interval of a reference image update in a reference image updating section. 
[Claim 7] shape of a hand of a user in a picture picturized with an image pick-up part at least — 
and - or with an operation recognition part which recognizes a motion, shape of a hand of a user 
who has recognized by said operation recognition part - and - or it having an indicator which 
displays a motion and with a contour extracting means which extracts a user's picturized outline. 
Outline waveform operation part which pursues said outline and calculates relation between an 
angle of a border line, and the length of a border line, i.e., an outline waveform, An interface 
device which filtered an outline waveform calculated by said outline waveform operation part, 
formed a shape filter which generates a shape waveform showing predetermined shape, and 



constituted said operation recognition part. 

[Claim 8]The interface device according to claim 7 it was made to make a user's operation judge 
based on a shape waveform which constituted two or more shape filters with several band pass 
filters in which zones differ, and was generated with said two or more shape filters. 
[Claim 9]The interface device according to claim 7 which constituted two or more shape filters 
at least with a band pass filter of outline wave shape corresponding to unevenness of a hand, and 
a band pass filter of outline wave shape corresponding to unevenness of a finger. 
[Claim 10] A coordinate table which makes correspondence of coordinates of a user's picturized 
contour shape and an outline waveform calculated by outline waveform operation part memorize 
is provided, The interface device according to claim 7 which provided coordinates operation part 
which calculates coordinates in which predetermined shape in an image pick exists using a wave 
height existence position and said coordinate table of a shape waveform, and constituted an 
operation recognition part. 

[Claim ll]The interface device according to claim 7 provides a shape judgment part which 
counts a pulse number in a shape waveform generated with a shape filter, and constitutes an 
operation recognition part, and it was made to make objective shape judge with an output value 
of said shape judgment part. 

[Claim 12]The interface device according to claim 7 which formed a differentiator which 
differentiates a shape waveform generated with a shape filter, and constituted an operation 
recognition part. 

[Claim 13] An interface device comprising: 
A displaying means. 

An input means which changes a position and shape of cursor which are displayed on said 
displaying means. 

A cursor memory measure which memorizes shape of a representative point coordinate 
representing a position of said cursor, and said cursor. 

An object memory measure which memorizes shape of a representative point coordinate 
representing a position of display things other than said cursor, and said display object, A 
position and shape of a display thing which are memorized to a position and shape, and said 
object memory measure of cursor memorized to said cursor memory measure are used, Have an 
interaction judging means which judges an interaction between said cursor and said display 
thing, and said interaction judging means, A distance calculation means to calculate distance 
between a representative point of said cursor of at least one point, and a representative point of 
said display thing of at least one point, A comprehensive judging means which opts for an 
interaction of said cursor and said display thing using a recognition result of an operation 
recognition means to recognize movement of said cursor, or change of shape, and distance which 
said distance calculation means calculates and said operation recognition means. 

* 

[Claim 14]The interface device according to claim 13 when an operation recognition means has 
recognized operation registered beforehand, wherein an interaction judging means generates an 
interaction to a display thing below a standard which distance which a distance calculation 
means calculates established beforehand. 

[Claim 15]Establish a movement vector calculating means which computes the move direction 
and movement magnitude of cursor in an operating space, and an interaction judging means is 
constituted, The interface device according to claim 13, wherein an interaction judging means 
opts for an interaction of said cursor and a display thing based on the move direction of cursor 



and movement magnitude of cursor which said move direction calculating means computes. 

[Claim 16]The interface device according to claim 15 when movement magnitude of cursor 

which a movement vector calculating means computes is below a reference value defined 

beforehand, wherein an interaction judging means generates an interaction. 

[Claim 17]The interface device according to claim 15, wherein an interaction judging means 

generates an interaction to a display thing which exists near the extension wire of the move 

direction of cursor which a movement vector calculating means computes. 

[Claim 18]The interface device according to claim 13 when shape of cursor and shape of a 

display thing become the combination registered beforehand, wherein an interaction judging 

means generates an interaction. 

[Claim 19]Have a shape judging means which recognizes cursor shape and shape of a display 
thing, and an interaction judging means is constituted, The interface device according to claim 
13, wherein an interaction judging means generates an interaction about a case where shape of 
cursor and shape of a display thing which said shape recognition means recognizes are in 
agreement. 

[Claim 20]The interface device according to claim 13 characterized by an interaction judging 
means generating an interaction when operation which an operation recognition means registered 
beforehand has been recognized to a display thing near the extension wire top of a look which 
establishes a look input means which detects a sight line direction, and said look input means 
detects. 

[Claim 21]The interface device according to claim 20 characterized by an interaction judging 
means generating an interaction when operation which cursor existed near the extension wire top 
of said look, and an operation recognition means registered beforehand has been recognized to a 
display thing near the extension wire top of a look which a look input means detects. 
[Claim 22] When an interaction is generated, a learning means which learns physical relationship 
of cursor and the target display thing, shape of said cursor, and shape of said display thing is 
established, The interface device according to claim 13, wherein an interaction judging means 
opts for an interaction based on a learning result of a learning means. 
[Claim 23]The interface device according to claim 22 characterized by an interaction judging 
means generating an interaction when physical relationship of cursor and the target display thing 
or shape of said cursor, and shape of said display thing are similar with physical relationship or 
shape which a learning means learned in the past. 

[Claim 24]The interface device according to claim 13 establishing a coordinate transformation 
means which performs coordinate conversion to an input from a cursor storage parts store and an 
object storage parts store to a distance calculation means, and constituting an interaction judging 
means. 

[Claim 25]The interface device according to claim 24 performing coordinate conversion so that 
physical relationship of cursor and the target display thing may be made to approach when an 
interaction is generated. 



[Translation done.] 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to the interface device which outputs and inputs the 
apparatus which has a display of information machines and equipment, such as a computer and a 
word processor, television, etc. 
[0002] 

[Description of the Prior Art]There are some which are performed by displaying cursor on the 
coordinates position detected with the mouse in a display screen although add some another 
information to the information in a display, or the information on a display is changed into it as 
this kind of a conventional interface device or being chosen. 

[0003] Drawing 30 shows the outline of this kind of conventional interface device. In drawing 30 % 
501 is a host computer, 502 is a display, and the manual operation button 503,504,505 of 
imagination is shown by the host computer 501 during the display 502. 506 is a mouse cursor, 
and based on the movement magnitude of the mouse 507 detected with the mouse 507, the host 
computer 501 carries out display control so that it may move all over a screen synchronizing 
with a motion of the mouse 507. By a user's moving the mouse cursor 506 to the position of the 
manual operation button of the arbitrary imagination in a display screen by moving the mouse 
507, and pushing the switch 508 on the mouse 507, A manual operation button is chosen and 
directions of operation can be given to the host computer 501 now. 
[0004] 

[Problem(s) to be Solved by the Invention]However, in the above-mentioned conventional 
composition, apart from the main part of apparatus, an input device called a mouse is required, 
and the pedestal of the table etc. which operate a mouse is required, and using for a portable 
information device etc. is not suitable. In order to operate it via a mouse, it is not necessarily a 
direct and intelligible interface. 

[0005]In view of this point, this invention does not need input devices, such as a keyboard and a 
mouse, but operation of apparatus is simply possible for it, and also its operativity is better, and it 
tends to provide a direct and intelligible interface device. 
[0006] 

[Means for Solving the Problem]What this invention is an interface device characterized by 
comprising the following, and can control information displayed on a screen only by changing 
shape of a hand. 

A recognition means to recognize shape of an operator's hand. 

A displaying means displayed on a screen by making into special shape the feature of shape of a 
hand recognized by this recognition means. 

A control means which controls information displayed in a screen by special shape displayed on 
a screen by this displaying means. 

[0007]An interface means with still more sufficient operativity is provided by recognizing a 
motion of a hand. A frame memory which saves a picture which picturized shape of a hand, or a 
motion at the motion recognition, A reference picture memory which accumulates a picture 
picturized at time before a picture saved in said frame memory as a reference image is provided, 
and it attains by extracting a difference between a picture in a frame memory, and a reference 
image accumulated into a reference picture memory, shape of a hand of a user in a picture 
picturized as other identification methods - and — or a motion was extracted as a user's outline, 
the outline was pursued, and it has attained by generating a shape waveform which calculates 



and filters relation between an angle of a border line, and the length of a border line, i.e., an 
outline waveform, and expresses predetermined shape. 

[0008] A cursor display means for displaying on a screen by making the feature of shape of a 
hand into special shape, and operating it as cursor, A means to memorize relation with display 
things other than the cursor display as a representative point coordinate and shape of representing 
a position of display things other than a cursor display, A means which carries out the operation 
judging of the interaction between a cursor display and said display thing is formed, and when 
imagination manipulator-izes a cursor display and it displays it, and holding a displayed virtual 
body, operation can be smoothly realized by an interaction in alignment with an operator's 
intention. 

[0009]It becomes controllable [ an interface device constituted as mentioned above / an icon 
display for what is called menu manipulation was made in a screen, and special shape which is 
equivalent to shape of a hand when a user goes to a recognition means for example, a hand is 
held up followed the icon display to the ability to be ]. 

[0010]Or if directions are given according to a gesture, special shape with which a given gesture 
was set up on a display screen corresponding to shape of a hand will be displayed, and. A virtual 
switch etc. which the motion was also displayed, for example, were displayed on a display screen 
can be chosen according to a gesture, or it can hold [ for the purpose of a display thing displayed 
on a screen ], or can carry, and input devices, such as a mouse, are not needed, but operation of 
very simple apparatus is attained. 

[001 l]The special shape set up corresponding to shape of a hand operates as a virtual 
manipulator in addition to mere cursor, An interface whose operativity improved further can 
realize an interaction with a display thing to operate with a virtual manipulator by making 
automatically a judgment which suited an operator's operation intention one by one. . 
[0012] 

[Embodiment of the Invention] 

(A 1st embodiment) A 1st embodiment of this invention, The displaying means which displays 
the feature of the shape of the hand recognized by a recognition means and these recognition 
means, such as an imaging device, in order to recognize the shape of an operator's hand on a 
screen as special shape by an icon etc., It is the interface device provided with the control means 
which controls the information which operated special shape, such as an icon displayed on the 
screen by this displaying means, as what is called cursor, and was displayed in the screen by 
changing the shape of a hand. 

[0013](A 2nd embodiment) A 2nd embodiment of this invention, the shape of the object in the 
picture picturized with the image pick-up part at least - and — or with the operation recognition 
part which recognizes a motion, the shape of the object recognized by this operation recognition 
part - and — or the indicator which displays a motion constituting and with the frame memory 
which saves the picture picturized in said image pick-up part. Provide the reference picture 
memory which accumulates the picture picturized at the time before the picture saved in this 
frame memory as a reference image, and in an operation recognition part The picture in a frame 
memory, It is the interface device which provided the image change extraction part which 
extracts a difference with the reference image accumulated into a reference picture memory. 
[0014](A 3rd embodiment) A 3rd embodiment of this invention, the shape of the hand of the user 
in the picture picturized with the image pick-up part at least — arid — or with the operation 
recognition part which recognizes a motion, the shape of the hand of the user who has 
recognized by this operation recognition part - and - or it having an indicator which displays a 



motion and with the contour extracting means which extracts a user's picturized outline. The 
outline waveform operation part which pursues this extracted outline and calculates the relation 
between the angle of a border line, and the length of a border line, i.e., an outline waveform, It is 
the interface device which filtered the outline waveform calculated by this outline waveform 
operation part, formed the shape filter which generates the shape waveform showing 
predetermined shape, and constituted said operation recognition part. 

[0015]If a user goes to the image pick-up part of the interface device constituted as mentioned 
above, for example, directions are given according to a gesture, an image pick-up part will 
picturize a user's picture. A contour extracting means extracts the outline of a user's picture, and 
this outline is changed as the angle of the border line to the horizon, i.e., an outline waveform, by 
setting a horizontal axis as the length of the border line which made the reference point on an 
outline the starting point by outline waveform operation part. This outline waveform is changed 
into the shape waveform which expresses the uneven shape of a finger with the shape filter 
constituted with the band pass filter of the predetermined zone, for example, the band pass filter 
equivalent to unevenness of a finger, While calculating a position with a hand, the number of the 
finger currently sent only by counting the pulse number which exists in this shape waveform, 
i.e., the shape of a hand, can be made to judge correctly. The virtual switch etc. which the given 
gesture was displayed on the display screen of an indicator based on a position of this sort and 
shape, for example, were displayed on the display screen can be chosen according to a gesture, 
and input devices, such as a mouse, are not needed, but operation of very simple apparatus is 
attained. 

[0016]Several band pass filters in which zones differ may constitute two or more shape filters, 
and based on the shape waveform generated with said two or more shape filters, it may constitute 
so that a user's operation may be made to judge. Thereby, two or more shape can be recognized 
now. 

[0017]The band pass filter of the outline wave shape corresponding to unevenness of a hand and 
the band pass filter of the outline wave shape corresponding to unevenness of a finger may 
constitute two or more shape filters at least. It is changed into the smooth shape waveform only 
reflecting unevenness of the portion of a hand by this, and is changed into the shape waveform 
only reflecting unevenness of ****. 

[0018]The coordinate table which makes corresjpondence of the coordinates of a user's picturized 
contour shape and the outline waveform calculated by outline waveform operation part 
memorize is provided, The coordinates operation part which calculates the coordinates in which 
the predetermined shape in an image pick exists may be provided using the wave height 
existence position and said coordinate table of a shape waveform, and an operation recognition 
part may be constituted. Thereby, the coordinates of contour shape calculate and the coordinates 
are outputted. 

[0019]The shape judgment part which counts the pulse number in the shape waveform generated 
with the shape filter is provided, an operation recognition part is constituted, and it may be made 
to make objective shape judge with the output value of said shape judgment part. The judgment 
of whether to be the shape which grasped the hand for whether it is the shape of 2 fingers with 
the pulse number by this is made easily. 

[0020]The differentiator which differentiates the shape waveform generated with the shape filter 
may be formed, and an operation recognition part may be constituted. By differentiation, a 
waveform becomes pulse form more and it becomes easy to count it of a pulse number. 
[0021] (A 4th embodiment) Although the old embodiment has described the example which 



operates the secondary screen displayed on the display screen, it requires this embodiment for 
the operation at the time of displaying a virtual three dimension on a two-dimensional display 
screen. In the virtual three-dimensional space generally displayed, if the operation which holds 
the virtual body in virtual space using cursor is assumed, it will become the following 
composition. 

[0022]As for a cursor-coordinates storage parts store and A3, in drawing 2L Al is [ a display 
and A5 of an object coordinate storing part and A4 ] contact judgment parts an input device and 
A2. Drawing 22 shows the cursor of the manipulator shape of two fingers which can be 
displayed from the shape of an operator's hand like the already described embodiment. The state 
where the state and drawing 22 (b), (d), and (f) to which drawing 22 (a), (c), and (e) opened the 
finger closed the finger is shown. Drawing 23 shows the example of the virtual body in virtual 
space. An operator assumes here the operation which holds the virtual body in virtual three- 
dimensional space using the cursor of two fingers. Drawing 24 (a) and drawing 24 (b) show the 
cursor in virtual space when holding a virtual body using cursor, and arrangement of a virtual 
body. Drawing 25 shows the display of display A4. 

[0023]If an operator's operation is given to the input part Al, the interval of the cursor 
coordinates memorized to the cursor-coordinates storage parts store A2 and two fingers of cursor 
will be updated based on said operation. Display A4 draws the virtual space which contains 
cursor and a virtual body using the position information on the virtual body which the 
information which the cursor-coordinates storage parts store A2 memorizes, and object 
coordinate storing part A3 memorize. Here, contact judgment part A5 calculates whether a 
virtual body contacts cursor in virtual space using the position information on the virtual body 
which the information which the cursor-coordinates storage parts store A2 memorizes, and 
object coordinate storing part A3 memorize. If the distance in the virtual space between two or 
more fields which constitute cursor and a virtual body is specifically calculated about between 
each field and a virtual body contacts between two fingers of cursor, it will judge that cursor has 
held the virtual body and objective coordinates will be henceforth changed in accordance with 
movement of cursor. 

[0024] However, by such a technique, even when in arrangement as shown in drawing 24 (a) and 
(b) the display by a display becomes like drawing 25 and the position of cursor and a virtual 
body is not thoroughly in agreement in virtual space, an operator may incorrect-judge that 
coordinates are in agreement. It becomes difficult from the difference in the depth perception in 
real space and virtual space smooth to also operate a display which wrote together the case where 
a three-dimensional display device is used, and drawing 24 (a) and (b). 

[0025] According to thus, the difference between the depth perception in the virtual space which 
is an operating space, and the depth perception in real space and the difference between a motion 
of the cursor which an operator means, and a motion of actual cursor. The interaction (a virtual 
body is held in the aforementioned example) which met an operator's intention in the interaction 
(for example, when holding a virtual body with the manipulator of imagination etc.) of cursor 
and the virtual body in virtual space cannot be realized smoothly. 

[0026]In this embodiment, in virtual space, operativity is improved by cursor control in non- 
contact [ with a gesture etc. ] in an operator, and. Only the distance between the cursor in virtual 
space and the component (it is the surface when virtual space is a three dimension) of a virtual 
body does not determine the existence of generating of an interaction with the virtual body in 
virtual space, The judgment of whether cursor causes a virtual body and an interaction can be 
brought close to an operator's intention, and the operativity of an interface can be further raised 



because an interaction judging means causes an interaction also to the subject with which the 
distance in virtual space is not necessarily close. It becomes possible to cause an interaction also 
to the subject with which the distance in virtual space is not necessarily close. 
[0027]The input means by which the 1st composition of this embodiment changes the position 
and shape of cursor of displaying on a displaying means and said displaying means, The cursor 
memory measure which memorizes the shape of the representative point coordinate representing 
the position of said cursor, and said cursor, The object memory measure which memorizes the 
shape of the representative point coordinate representing the position of display things other than 
said cursor, and said display object, The position and shape of a display thing which are 
memorized to the position and the shape, and said object memory measure of the cursor 
memorized to said cursor memory measure are used, Constitute from an interaction judging 
means which judges the interaction between said cursor and said display thing, and said 
interaction judging means, A distance calculation means to calculate the distance between the 
representative point of said cursor of at least one point, and the representative point of said 
display thing of at least one point, It is an interface device comprising an operation recognition 
means to recognize movement of said cursor, or change of shape, and the comprehensive judging 
means which opts for the interaction of said cursor and said display thing using the recognition 
result of the distance which said distance calculation means calculates, and said operation 
recognition means. 

[0028]By this composition, the existence of generating of the interaction of the cursor which an 
operator operates in virtual space, and the virtual body in virtual space, Only the distance 
between said cursor in virtual space and the component (it is the surface when virtual space is a 
three dimension) of said virtual body does not determine, When a comprehensive judging means 
judges the existence of generating of an interaction by operation of the cursor which the distance 
and the operation recognition means during the representative point which a distance calculation 
means computes recognize, it becomes possible to cause an interaction also to the subject with 
which the distance in virtual space is not necessarily close." 

[0029]In the 1st composition, when the operation which the operation recognition means 
registered beforehand has been recognized, it is good also as the 2nd composition an interaction 
judging means generates [ composition ] an interaction to the display thing below the standard 
which the distance which a distance calculation means calculates established beforehand. 
[0030]In the 1st and 2nd composition, establish the movement vector calculating means which 
computes the move direction and movement magnitude of cursor in an operating space, and an 
interaction judging means is constituted, It is good also as the 3rd composition as which an 
interaction judging means determines the interaction of said cursor and a display thing based on 
the move direction of cursor and the movement magnitude of cursor which said move direction 
calculating means computes. 

[0031]In the 3rd composition, when the movement magnitude of the cursor which a movement 
vector calculating means computes is below the reference value defined beforehand, an 
interaction judging means is good also as the 4th composition that generates an interaction. 
[0032]In the 3rd and 4th composition, it is good also as the 5th composition that an interaction 
judging means makes generate an interaction to the display thing which exists near the extension 
wire of the move direction of the cursor which a movement vector calculating means computes. 
[0033]In the 1st - the 5th composition, when the shape of cursor and the shape of a display thing 
become the combination registered beforehand, an interaction judging means is good also as the 
6th composition that generates an interaction. 



[0034]In the 1st - the 6th composition, have a shape judging means which recognizes cursor 
shape and the shape of a display thing, and an interaction judging means is constituted, An 
interaction judging means is good also as the 7th composition that generates an interaction about 
the case where the shape of cursor and the shape of a display thing which said shape recognition 
means recognizes are in agreement. 

[0035]In the 1st - the 7th composition, when the operation which the operation recognition 
means registered beforehand has been recognized to the display thing near the extension wire top 
of the look which establishes the look input means which detects a sight line direction, and said 
look input means detects, an interaction judging means is good also as the 8th composition that 
generates an interaction. 

[0036]In the 8th composition, when the operation which cursor existed near the extension wire 
top of said look, and the operation recognition means registered beforehand has been recognized 
to the display thing near the extension wire top of the look which a look input means detects, an 
interaction judging means is good also as the 9th composition that generates an interaction. 
[0037]In the 1st - the 9th composition, when an interaction is generated, the learning means 
which learns the physical relationship of cursor and the target display thing, the shape of said 
cursor, and the shape of said display thing is established, An interaction judging means is good 
also as the 10th composition that opts for an interaction based on the learning result of a learning 
means. 

[0038]In the 10th composition, when the physical relationship of cursor and the target display 
thing or the shape of said cursor, and the shape of said display thing are similar with the physical 
relationship or shape which the learning means learned in the past, an interaction judging means 
is good also as the 11th composition that generates an interaction. 

[0039]In the 1st - the 1 1th composition, it is good also as the 12th composition that establishes 
the coordinate transformation means which performs coordinate conversion to the input from a 
cursor storage parts store and an object storage parts store to a distance calculation means, and 
constitutes an interaction judging means. 

[0040]In the 12th composition, when an interaction is generated, it is good also as the 13th 
composition that performs coordinate conversion so that the physical relationship of cursor and 
the target display thing may be made to approach. 

[0041](The 1st example) Drawing 1 shows the appearance of the 1st example of the interface 
device by this invention. In drawing 1 , the display which uses 1 for a host computer and 2 uses 
for a display, and 3 are CCD cameras which picturize a picture. CCD camera 3 arranges the 
imaging surface to the display direction and uniform direction of the display 2, and when a user 
goes to a display surface, it can picturize the shape of a user's note. On the display 2, the icon 
200 which performs the display reflecting the menu 201,202 and the shape of said note can be 
displayed now. 

r00421 Drawing 2 shows the detailed block diagram of this example. The picture inputted from 
CCD camera 3 is stored in the frame memory 21. At the reference picture memory 25, the 
picture of the background which does not contain the person picturized beforehand is saved as a 
reference image. A reference image can be updated as required if needed. 
[0043]The shape identification device 22 removes a background image out of a picture by 
extracting the difference of the picture stored into the frame memory 21, and the picture saved at 
the reference picture memory 25, and. For example, the portion equivalent to a user's hand is 
extracted and it is judged that it is the shape of 2 fingers where it is shown in drawing 3 (b) again 
whether the shape is the shape of 1 finger as shown, for example in drawing 3 (a), or whether 



other than this, it is drawing 3 (c). 

f 00441 Drawing 4 is what showed the detailed embodiment of the shape identification device 22, 
and shows the example constituted by the picture difference operation part 221, the outline 
extraction part 222, and the shape identification part 223. The picture difference operation part 
221 calculates the difference of the picture stored into the frame memory, and the picture saved 
at the reference picture memory as mentioned above. Thereby, the portions of an object, for 
example, a user, to extract and a background are separable. For example, when the picture 
difference operation part 221 is constituted from a simple subtracting circuit, as shown in 
drawing 5 , only the portion of the hand of the user in the picture in a frame memory can be 
extracted. The outline extraction part 222 extracts the contour shape of the object which exists in 
the picture of the result calculated by the picture difference operation part 221. As a concrete 
example of a method, contour shape can be easily extracted by extracting the edge of a picture. 
[0045]It is judged whether the shape identification part 223 is the shape of 2 fingers where it is 
shown in drawing 3 (b) again whether it is the shape of 1 finger as carries out detailed 
discernment of the contour shape of a hand, for example, the shape shows to drawing 3 (a) 
extracted by the outline extraction part 222. It is possible to use template matching, the matching 
technique with a geometric model, a neural network, etc. as a method of identifying shape. 
[0046]The icon generation part 24 generates an icon image as special shape which should be 
displayed on a display based on the discriminated result of the shape of the hand by the shape 
identification part 223. for example, — a hand — shape ~ a discriminated result — one — a ** — a 
finger — shape — it is ~ if — for example - drawing 6 — (— a — ) ~ being shown ~ as - a number 

- " - one -- " — an icon — generating - moreover — two — a ** — a finger - shape ~ it is - if ~ 
for example — drawing 6 -- (-- b --) - being shown — as - a number - " — two — " — an icon — 
generating . A 2 finger type icon if it is a 1 finger type icon if the discriminated result of the 
shape of a hand is the shape of 1 finger as shape of an icon, as shown iri drawing 6 (c) the shape 
of 2 fingers again, as show drawing 6 (d) etc. may be generated. The display control part 23 
controls a display based on the discriminated result of the shape of the hand by the shape 
identification part 223. For example, while displaying said icon based on a discriminated result, 
it displays by emphasizing the menu beforehand matched with the discriminated result based on 
a note-like discriminated result. 

[0047]The operation instances by this example constituted as mentioned above are shown below. 
If a user faces to apparatus provided with the interface device of this example and a hand is made 
into the shape of 1 finger, as shown in drawing 7 (a), while the icon of number" 1" will be 
displayed on a display, highlighting of the display of the television which is the 1st menu is 
carried out. It is possible to call an operator's attention further with outputting a sound or a sound 
from a display device in accordance with highlighting at this time. Here, if a hand is made into 
the shape of 2 fingers like drawing 7 (b), while the icon of number"2" will be displayed on a 
display, highlighting of the display of the network which is the 2nd menu is carried out. this state 

- for example, fixed time — the 2nd menu is selected by maintaining the shape of the same hand, 
and instructions can be given so that a network terminal may be displayed on a host computer. A 
sound etc. may be used together for selection of a menu. In except the shape of the hand 
beforehand decided like drawing 7 (c), an icon and a menu are not displayed on a display, and a 
command is not sent to a host computer. 

[0048]Thus, according to this example, it is possible to identify the shape of the hand in the 
picturized picture and to control apparatus, such as a computer, according to a discriminated 
result, and it is possible to operate it by non-contact from remoteness, without using apparatus, 



such as a keyboard and a mouse. Positive operation is attained that it can be operated while a 
user checks a discriminated result by making the result of having identified the shape of the hand 
reflect in a screen, and it is easy to use. 

[0049] Although this example showed the example only applied to menu selection, when an icon 
display sets ****** as shape with a hand beforehand replace a picture, a text, etc., it cannot be 
overemphasized that it is also easy to carry out writing control of a picture or the character to a 
display. 

[0050] (The 2nd example) Drawing 8 shows the appearance of the 2nd example of the interface 
device by this invention. In drawing 8 , the same numerical value shows a component to the 
portion which has the same function as the 1st example. That is, the display which uses 1 for a 
host computer and 2 uses for a display, and 3 are CCD cameras which picturize a picture. CCD 
camera 3 arranges the imaging surface to the display direction and uniform direction of the 
display 2, and when a user goes to a display surface, it can picturize a user's gesture. On the 
display surface of the display 2, the icon 203 of the arrow cursor which chooses the virtual 
switches 204, 205, and 206 and said virtual switch can be displayed now. 
[00511 Drawing 9 shows the block diagram of the detailed composition of this embodiment. The 
picture inputted from CCD camera 3 is stored in the frame memory 21. At the reference picture 
memory 25, the picture picturized beforehand is saved as a reference image. The timer 261 and 
the picture updating section 262 constitute the reference image updating section 26, it transmits 
the newest image saved at the frame memory 21 with the fixed time interval directed with the 
timer 261 to the reference picture memory 25, and updates a reference image. 
[0052]The operation recognition part 22 removes a background image out of a picture by 
extracting the difference of the picture stored into the frame memory, and the picture saved at the 
reference picture memory, and. For example, the portion equivalent to a user's hand is extracted 
and it is judged whether it is the shape of a fist where it is shown in drawing 10 (b) again 
whether the shape is the shape of 1 finger as shown, for example in drawing 10 (a). 
[00531 Drawing 1 1 is what showed the details of the operation recognition part 22, and shows the 
example constituted by the picture difference operation part 221, the outline extraction part 222, 
the shape change identification part 225, and the position detector 224. 
[0054]The picture difference operation part 221 calculates the difference of the picture stored 
into the frame memory 21 as mentioned above, and the picture saved at the reference picture 
memory 25. While the portion of the hand of an object, for example, a user, to extract as a 
motion and the portion of a background are separable by this, only the object images which are 
moving can be extracted. For example, when the picture difference operation part 221 is 
constituted from a simple subtracting circuit, as shown in drawing 12 , only the portion of the 
hand in a reference image and the portion of the hand in the newest image in a frame memory 
can be extracted, and only the portion of the hand which is moving can be identified easily. The 
outline extraction part 222 extracts the contour shape of the portion of the hand the object which 
exists in the picture of the result calculated by the picture difference operation part 221, i.e., 
before moving, and after moving. As a concrete example of a method, contour shape can be 
easily extracted by extracting the edge of a picture. 

[0055]It is judged whether the shape change identification part 225 is the shape of a fist where it 
is shown in drawing 10 (b) again whether it is the shape of 1 finger as carries out detailed 
discernment of the contour shape of the portion of the hand after moving, for example, the shape 
shows to drawing 10 (a) extracted by the outline extraction part 222. The position detector 224 
calculates the barycentric coordinates of the contour shape of the portion of the hand of the user 



after [ said ] moving simultaneously. 

[0056]The icon generation part 24 generates the icon image which should be displayed on a 
display based on the discriminated result of the shape of the hand by the shape change 
identification part 225. The icon of an arrow if the discriminated result of the shape of a hand is 
the shape of 1 finger as an example of an icon image, as shown, for example in drawing 13 (a) is 
generated, and if it is the shape of a fist, the icon of X seal as shown, for example in drawing 13 
(b) will be generated. It is good also as composition which generates the icon of the shape which 
imitated the shape of 2 fingers as shown, for example in drawing 13 (c) when the discriminated 
result of the shape of a hand was the shape of 2 fingers, and generates the icon of the shape 
which imitated the shape of the fist as shown, for example in drawing 13 (d) when it was the 
shape of the fist. 

[0057]The display control part 23 controls the display position on the display 2 of the icon 
generated by the icon generation part 24, and comprises the coordinate transformation part 231 
and the coordinates pars inflexa 232. The coordinate transformation part 231 performs 
conversion to the display coordinates of the display 2 from the coordinates of the picturized 
picture, and the coordinates pars inflexa 232 reverses the right-and-left position of the changed 
display coordinates. 

[0058]That is, conversion to the display coordinates of the display 2 from the barycentric 
coordinates in the picture of the portion equivalent to the hand of the user who detected by the 
position detector 224 is performed, coordinates on either side are reversed, and an icon is 
displayed on the display 2. By this operation, if a user moves a hand to the right, the same with 
having copied this operation in the mirror, an icon will go on a display screen and it will move to 
the right. 

[0059]The operation instances by this example constituted as mentioned above are shown below. 
As shown in drawing 8 , a user faces to apparatus provided with the interface device by this 
example, and if a hand is made into the shape of 1 finger and moved, the arrow cursor displayed 
on the display will move to the arbitrary positions corresponding to a motion of a hand. Next, if 
arrow cursor is moved, a hand is grasped and it is made fist shape by moving a hand after the 
arbitrary virtual switches 204,205,206 displayed on the display 2, The virtual switch 204,205,206 
is chosen and instructions can be given to the host computer 1. 

[0060] Although it had composition which recognizes the shape of the object in the picturized 
picture, and a motion in this example, it cannot be overemphasized that it is good also as 
composition which recognizes the shape of the object in the picturized picture or either of the 
motions. 

[0061]the shape of the object in the picture which was picturized as mentioned above according 
to this example — and — or with the operation recognition part which recognizes a motion, the 
shape of the object recognized by the operation recognition part — and — or with the indicator 
which displays a motion. Provide the reference picture memory which accumulates the picture 
picturized at the time before the picture saved in the frame memory which saves the picture 
picturized in the image pick-up part, and the frame memory as a reference image, and in an 
operation recognition part The picture in a frame memory, By extracting a difference with the 
reference image accumulated into a reference picture memory, If a user goes to an image pick-up 
part, for example, directions are given according to a gesture, the virtual switch etc. which the 
given gesture was displayed on the display screen, for example, were displayed on the display 
screen can be chosen according to a gesture, and input devices, such as a mouse, will not be 
needed, but operation of very simple apparatus will be attained. 



[0062] (The 3rd example) The appearance of the 3rd example of the interface device by this 
invention is the same as that of drawing 8 shown in the 2nd example, and only a portion which 
explains the same portion as the 2nd example using drawing 8 and drawing 10 , in addition is 
different is shown after drawing 14 . 

r00631 Drawing 14 shows the detailed block diagram of the 3rd example. The picture inputted 
from CCD camera 3 is stored in the frame memory 31. It is judged whether the operation 
recognition part 32 is the shape of a fist where the portion which is equivalent to a user's hand, 
for example is extracted out of the picture stored into the frame memory 31, and the shape shows 
drawing 10 (b) whether it is the shape of 1 finger as shown, for example in drawing 10 (a). 
r00641 Drawing 15 shows the details of the operation recognition part 32, and explains the 
detailed operation using drawing 16 - drawing 20 . First, the outline extraction part 321 extracts 
the contour shape of the object which exists in a picture. Contour shape can be easily extracted 
by carrying out binarization of the picture and extracting the edge as a concrete example of a 
method. Drawing 17 (a) is an example of the extracted border line, and the portion of the hand 
shows the shape of the 1 finger. 

[0065]Next, as a border line is followed in the direction of the arrow of a figure 
(counterclockwise rotation) with the starting point s as the starting point in a figure and the 
contour shape of the object extracted by the outline extraction part 321 as shown in drawing 17 
(a) is shown in drawing 19 , the outline waveform operation part 322, The angle theta from the 
horizon of the border line in each point x on a border line is extracted as a function over the 
distance 1 from the starting point s, The coordinates of each point on the border line 
corresponding to the distance 1 are used as a table, and it saves at the translation table 324 at the 
same time the bottom changes wholly the distance 1 as shown in drawing 17 (b) into wave shape 
as a time-axis. The shape filter 1 and the shape filter 2 which are shown in 3231 and 3232 are a 
filter which passes the zone corresponding to unevenness of the portion of the hand in the outline 
waveform shown in drawing 17 (b), respectively, and unevenness of the portion of a finger. 
[0066] With the shape filter 1, as shown in drawing 17 (c), drawing 17 (b), Only unevenness of 
the portion of a hand is changed into the reflected smooth shape waveform, and with the shape 
filter 2. As shown in drawing 17 (d), it is changed into the shape waveform only reflecting 
unevenness of the finger, and differentiates with the differentiators 3251 and 3252, respectively, 
and the shape differential waveform shown in drawing 17 (e) and drawing 17 (f) is obtained 
eventually. Judging whether the shape judgment part 3262 is the shape of a fist where it is shown 
in drawing 10 (b) again whether the contour shape of the portion of a hand is the shape of 2 
fingers as shown in drawing 10 (a), the coordinates operation part 3261 calculates the barycentric 
coordinates of the contour shape of the portion said user's hand simultaneously. Ask for the 
coordinates operation part 3261 and the position in which the big pulse shape in the shape 
differential waveform shown in drawing 17 (e) exists, lcl, and lc2 with the coordinate 
conversion table 324. It changes into cl point shown in drawing 20 , and c2 point, and from the 
border line of the portion of the hand from cl point to c2 point, the center of gravity of the 
portion of a hand is calculated and it outputs as hand coordinates. 

[0067]The shape judgment part 3262 counts and outputs the number of the pulse shapes with 
which the portion of the finger in the shape differential waveform of drawing 17 (f) corresponds. 
That is, since two big pulse shapes equivalent to the portion of a finger exist, in the case of 
drawing 17 (f), it judges with it being the shape of 2 fingers as shown in drawing 10 (a), and 
outputs to it. As shown in drawing 18 (a), in the case of the shape which grasped the hand, there 
is almost no unevenness of the portion of a finger and the output of the shape filter 2, As are 



shown in drawing 18 (c), and it becomes a shape waveform without unevenness, therefore the 
output of the differentiator 3262 is also shown in drawing 18 (d). It becomes a shape differential 
waveform which does not have a pulse shape, and it can be set to 0, and the count of a pulse 
shape judges with it being the shape of a fist as shown in drawing 10 (b), and can be outputted. A 
simple threshold approach, a neural network, etc. can be used as an example of concrete 
composition of this shape judgment part 3262. 

[0068]The icon generation part 34 of drawing 14 g enerates the icon image which should be 
displayed on a display based on the discriminated result of the shape of the hand by the shape 
judgment part 3262 in drawing 15 . For example, the icon of X seal if it is the shape of a fist the 
icon of an arrow if the discriminated result of the shape of a hand is the shape of 1 finger, as 
shown, for example in drawing 16 (a) again, as shows drawing 16 (b), for example is generated. 
The display control part 33 controls the display position on the display of the icon generated by 
the icon generation part 34, and comprises the coordinate transformation part 331 and the 
coordinates pars inflexa 332. The coordinate transformation part 331 performs conversion to the 
display coordinates of a display from the coordinates of the picturized picture, and the 
coordinates pars inflexa 332 reverses the right-and-left position of the changed display 
coordinates. That is, conversion to the display coordinates of a display from the barycentric 
coordinates in the picture of the portion equivalent to a user's hand which the coordinates 
operation part 3261 in drawing 15 outputted is performed, coordinates on either side are 
reversed, and an icon is displayed on a display. By this operation, if a user moves a hand to the 
right, the same with having copied this operation in the mirror, an icon will go by the shape of a 
display screen, and it will move to the right. 

[0069]The operation instances of this example constituted as mentioned above are shown below. 
As shown in drawing 8 , a user faces to apparatus provided with the interface device by this 
example, and if a hand is made into the shape of 1 finger and moved, the arrow cursor of the icon 
203 displayed on the display 2 will move to the arbitrary positions corresponding to a motion of 
a hand. Next, if arrow cursor is moved, a hand is grasped and it is made fist shape by moving a 
hand after the arbitrary virtual switches 204,205,206 displayed on the display 2, the virtual 
switch is chosen and instructions can be given to the host computer L 

[0070] As an example of the icon to display, if the shape of a hand itself is iconified to drawing 
16 (c) and (d) so that it may be shown, a motion and correspondence of a actual hand can be 
taken to it, and it is intelligible for it. The contour shape data of the hand which could register 
beforehand the picture as shown in drawing 16 (c) and (d), and was specifically extracted by the 
outline extraction part can be reduced or expanded to arbitrary sizes, and can also be used as an 
icon image. 

[0071]Thus, when a user goes to the image pick-up part of an interface device, for example, 
directions are given according to a gesture, this example an image pick-up part, A user's picture 
is picturized, the outline of a user's picture is extracted, and it is changed as the angle of the 
border line to the horizon, i.e., an outline waveform, by setting a horizontal axis as the length of 
the border line which made the reference point on an outline the starting point. This outline 
waveform is changed into the shape waveform which expresses the uneven shape of a finger 
with the shape filter constituted with the band pass filter of the predetermined zone, for example, 
the band pass filter equivalent to unevenness of a finger, While calculating a position with a 
hand, the number of the finger currently sent only by counting the pulse number which exists in 
this shape waveform, i.e., the shape of a hand, can be made to judge correctly. The virtual switch 
etc. which the given gesture was displayed on the display screen based on a position of this sort 



and shape, for example, were displayed on the display screen can be chosen according to a 
gesture, and input devices, such as a mouse, are not needed, but operation of very simple 
apparatus is attained. 

[0072](The 4th example) The 4th example of this invention is described with reference to 
drawings. Drawing 26 shows the block diagram of the 4th example of an interface device. 
[0073]In drawing 26 , 41 an input means and 42 a cursor memory measure and 43 An object 
memory measure, 44 - a displaying means and 45 - an interaction judging means and 45a — a 
distance calculation means and 45b — as for a coordinate transformation means and 46, a 
comprehensive judging means and 45 d of a shape judging means and 45 f are [ an operation 
recognition means and 45c / a movement vector calculating means and 45e ] look input means. [ 
of a learning means and 45 g ] 

[0074]In drawing 26 , an operator operates it to the input means 41, and based on said operation 
result, The representative point coordinate and shape where the cursor memory measure 42 
represents the position in the virtual space of cursor are changed and memorized, The displaying 
means 44 the position in the virtual space of the virtual body which the representative point 
coordinate and shape of representing the position in the virtual space of the cursor which the 
cursor memory measure 42 memorizes, and the object memory measure 43 memorize based on 
the representative point coordinate and shape to represent said cursor and said virtual body. It 
two-dimensions-displays or displays in three dimensions. 

[0075]The look input means 46 detects the position of the look of the operator on said display. 
The distance calculation means 45a calculates the distance in the virtual space of cursor and a 
virtual body by being based on the coordinates of the representative point memorized to the 
cursor memory measure 42 and the object memory measure 43. The operation recognition means 
45b recognizes operation operation based on the contents memorized to the cursor memory 
measure 42 and the object memory measure 43. The movement vector calculating means 45d 
computes the move direction and migration length of cursor in virtual space. It is judged whether 
the shape judging means 45e is suitable for the shape of cursor and the shape of a virtual body to 
cause an interaction. The learning means 45f memorizes said cursor when the comprehensive 
judging means 45c causes the interaction between cursor and a virtual body, the position of said 
virtual body, and the relation of shape, and outputs whether the present state is similar to the 
state where the interaction was caused in the past. 

[0076]The distance of a between [ the cursor to which the distance calculation means 45a outputs 
the comprehensive judging means 45c, and a virtual body ], The recognition result which the 
operation recognition means 45b has recognized, and the move direction and migration length of 
cursor which the movement vector calculating means 45d computes, The position of the look 
which the look input means 46 detects, and the decision result of the shape judging means 45e, It 
judges whether said cursor and said virtual body interact based on a similar degree with the 
interaction of the past which the learning means 45f outputs, and the representative point 
coordinate and shape of said cursor or said virtual body are changed according to the result of an 
interaction. The coordinate transformation means 45g performs coordinate conversion so that 
both position may approach the coordinates in the virtual space of the cursor and the target 
virtual body which the distance calculation means 45a uses for distance calculation, when the 
interaction judging means 45 causes an interaction. 

[0077]Drawing_22_(a) and (b) is a thing of the manipulator shape.of two fingers in a first 
embodiment of the cursor used for the interface device of this invention. In drawing 22 , the state 
to which (a) opened the finger, and (b) are in the state which closed the finger. Drawing 22 (c) 



1 



and (d) is a thing of the manipulator shape of 2 finger 2 joint in a second embodiment of the 
cursor used for the interface device of this invention. In a figure, the state to which (c) opened 
the finger, and (d) are in the state which closed the finger. Drawing 22 (e) and (f) is a third 
embodiment of the cursor used for the interface device of this invention, and is a thing of the 
shape of a note of five fingers. In drawing 22 , the state to which (e) opened the hand, and (f) are 
in the state which closed the hand. Drawing 23 (a) and (b) is an example of the object in the 
virtual space used for the interface device of this invention ((a) is a cube and (b) is a plane body). 
[0078]Operation of the interface device constituted as mentioned above is explained. This 
example assumes the operation which moves cursor like drawing 22 into three-dimensional 
virtual space, and an operator holds a virtual body like drawing 23 which exists in said virtual 
space, and moves. An operator's operation is performed to the input means 41. 
[0079]Here, the command input by the means or a camera, a keyboard, speech recognition as 
shown in drawing 27 (a) - (c), etc., etc. can be used for an input as an input device which inputs 
the information for changing the position or shape of cursor. Drawing 27 (a) is a mouse and 
operates cursor by movement of a mouse body, the click of a button, etc. Drawing 27 (b) is a 
data glove, equip people's hand with it, makes the position in the joint angle of a finger, or the 
real space of a data glove reflect in the position and shape of cursor, and operates cursor. 
Drawing 27 (c) is a joy stick and operates cursor according to operation of a lever, and 
concomitant use with a manual operation button. In using a camera, a part of body or body (hand 
etc.) is picturized with a camera, and it reads the shape and the position of a hand. 
r00801 Drawing 28 shows the 1 embodiment of the shape extraction at the time of picturizing 
only a hand with a camera. Drawing 28 (a) is the example which picturized the hand with the 
camera. It is drawing 28 (b) which carried out binarization of the luminosity of each pixel of the 
picture of drawing 28 (a). In drawing 28 (b), the judgment of the degree of opening and closing 
of a hand is attained with the ratio etc. of the length of the neighborhood of rectangular length 
and the length of the horizontal neighborhood which are circumscribed to a **** field, and the 
input of a position and distance is attained from the barycentric coordinates and area of the whole 
black pixel. The input means 41 outputs said contents of operation (the movement magnitude of 
cursor, a cursor-shaped changing amount, etc.) to the cursor memory measure 42. 
[0081]The cursor memory measure 42 memorizes the coordinates and shape in the virtual space 
of the representative point of the cursor memorized by a cursor memory measure based on the 
contents of operation which the input means 41 outputted. As a representative point, the 
barycentric coordinates (XO, YO, ZO) of cursor are used as a representative point. 
[0082] As a representative point, center coordinates of each field, coordinates of the peak, etc. 
which constitute cursor may be used. In the case of drawing 22 (a), in the interval d of two 
fingers, drawing 22 (b). and (c), let as shape inside thetan (the number whose n is a joint: when 
thetan becomes small, the state where a joint bends is shown) of the joint of each finger be 
memory information. As shape, the fingertip of each finger, the coordinates in the virtual space 
of each joint, etc. may be used. 

[0083]The object memory measure 43 memorizes the coordinates and shape of a representative 
point of a virtual body in the virtual space shown in drawing 23 which is the target of operation. 
As a representative point, they are barycentric coordinates (cube: (XI, Yl, Zl), plane body : (X2, 
Y2, Z2)) of a virtual body. 

It uses as a representative point. As a representative point, center coordinates of each field, 
coordinates of the peak, etc. which constitute a virtual body may be used. The parameter alpha 
showing the shape beforehand defined as shape is memorized as shape (a cube is defined as 



alpha= 1 and a plane body is defined as alpha= 2 here). Apex coordinates etc. may be used as 
shape. 

[0084]The displaying means 44 indicates the image at which it looked from the viewpoint which 
specified virtual space beforehand based on the cursor which the cursor memory measure 42 and 
the object memory measure 43 memorize, the position of a virtual body, and the information on 
shape by two-dimensional. The display example of a displaying means is shown in drawing 29 
(a). If an operator operates it, the display position or shape of cursor will change and an operator 
will continue operation based on the display. 

[0085]The interaction judging means 45 judges whether cursor has held the object, whenever the 
cursor position changes (existence of an interaction), and when it judges that cursor has held the 
object, the coordinates of a virtual body also make it move it in accordance with movement of 
cursor. The distance calculation means 45a calculates the barycentric coordinates (XI, Yl, Zl) 
of the virtual body memorized to the barycentric coordinates (XO, YO, ZO) and the object 
memory measure 43 of the cursor memorized to the cursor memory measure 42, and the distance 
of (X2, Y2, Z2). 

[0086]An operation recognition means 45b means recognizes operation of "holding" as operation 
registered beforehand, using change of the shape of cursor. The state the state where the interval 
d of two fingers continues decreasing "is held" in the case of the cursor of drawing 22 (a) where 
it is recognized as operation and angle thetan of all the fingers continues decreasing in the case 
of the cursor of drawing 22 (b) and (c) is recognized to be the operation "to hold." It is the time 
series pattern recognition technique (it and) about a serial change of the parameters (said d, 
thetan, etc.) with which shape is expressed as the recognition technique of operation. [ table-] 
After making specific operation learn beforehand using DP matching, a HIDOUN Markov model 
(HMM), a recurrent neural network, etc., it may use as the recognition technique. 
[0087]The movement vector calculating means 45d computes the move direction and movement 
magnitude of cursor in virtual space by using change of the barycentric coordinates (XO, YO, ZO) 
of cursor. For example, let the direction and size of a difference vector of the barycentric 
coordinates (XO, YO, ZO) t of the current time t, and the barycentric coordinates (XO, YO, ZO) t-1 
in front of 1 time be the move direction of cursor, and movement magnitude. 
[0088]The shape judging means 45e judges (whether it is suitable for the shape of cursor to 
cause a virtual body and an interaction) for whether it is shape with suitable shape of the cursor 
memorized to a cursor memory measure to hold the virtual body of the shape memorized to an 
object memory measure. Here, when the value of the parameter alpha showing objective shape is 
1, the state where the finger of cursor is open is changed into a suitable state. As judgment of the 
state where the finger of cursor is open, For example, in the case of the cursor of drawing 22 (a), 
it is considered as the case where the value of d is larger than the mean value of the maximum 
dmax of d, and 0, and, in drawing 22 (b) and (c), each joint angle thetan considers all [ mean 
value / of each maximum thetan max and 0 ] as the case where it is large. 
[0089]When the value of the parameter alpha showing objective shape is 0, the interval of the 
fingertip of cursor changes a narrow state into a suitable state. As judgment of the state where the 
fingertip of cursor is narrow, for example in the case of the cursor of drawing 22 (a). It is 
considered as the case where the value of d is smaller than the mean value of the maximum dmax 
of d, and 0, and, in drawing 22 (b) and (c), each joint angle thetan considers all [ mean value / of 
each maximum thetan max and 0 ] as the case where it is small. The parameter (d or thetan) 
showing cursor shape when it has held in the state where cursor contacts a virtual body in virtual 
space as a diagnosis of shape is memorized beforehand, If in agreement in the range whose value 



of each parameter is **30% It holds. It can perform judging that it is suitable for operation etc. 
[0090]The look input means 46 detects an operator's look, and computes the coordinates 
(coordinates of the point of regard) at which an operator gazes on the screen which the 
displaying means 44 displays. The point of regard on said screen is computed by measuring the 
position of an operator's head using a camera etc. using the method of detecting direction of an 
operator's pupil using a CCD camera, a photosensor, etc. as a detection means of a look etc. 
[0091]When the comprehensive judging means 45c judges the learning means 45f that cursor has 
held the virtual body. The relative physical relationship (vector which connects the center of 
gravity of cursor and the center of gravity of a virtual body) of the position of the parameter (d or 
thetan) showing the shape of ** cursor, the parameter alpha showing the shape of the held virtual 
body, and said cursor and the position of said virtual body is memorized, The physical 
relationship of the center of gravity of the parameter showing the shape of the present virtual 
body, the parameter showing the shape of the surrounding virtual body, and the present cursor 
and the center of gravity of the surrounding virtual body, When the object has been held in the 
past, in being near, it judges that it is similar with the past situation, and outputs 1 (for example, 
when the value of the element of each dimension of the vector showing each parameter and 
physical relationship is in agreement in the past value and **30% of range etc.), and in the case 
of others, 0 is outputted. The relative physical relationship of the position of the parameter 
showing the shape of the cursor in the case of having held the object, the parameter alpha 
showing the shape of the held virtual body, and said cursor and the position of said virtual body 
may be learned using a neural network etc. in the past as other learning means. It can also learn 
including the physical relationship etc. of the fixation point coordinates on the screen detected by 
the sight line detecting means 46 as an item to learn, and the coordinates on the display screen of 
cursor. 

[0092]When the object has been held, the coordinate transformation means 45g is transformed so 
that the distance of the cursor in virtual space and the target virtual body may approach the 
coordinates which a distance calculation means uses for distance calculation (when an interaction 
is caused). For example, when each coordinates when cursor has held the virtual body are (100, 
150, 20), and (105, 145, 50), a coordinate transformation means changes like [ Z coordinate / of a 
difference / largest ] (1) type among each axis of coordinates. 

[0093]Z'=0.8xz (1) — here, the cursor which, as for z, a coordinate transformation means 

considers as an input and the Z coordinate of the center of gravity of a virtual body, and Z' show 
the Z coordinate which a coordinate transformation means outputs. 

[0094]In this case, the value of an X coordinate and the value of a Y coordinate are not changed. 
Since the value memorized to a cursor memory measure and an object memory measure does not 
change, the screen which a displaying means displays does not change. If operation which an 
operator holds is performed by performing the above conversion even when the distance in 
virtual space is separated, the cursor in the case of performing distance calculation and the 
distance between virtual bodies will become small after it, and a distance calculation means will 
become possible [ calculating the distance near the depth perception which an operator perceives 

]. 

[0095]When the comprehensive judging means 45c is below the standard which the distance of a 
between [ the cursor which the distance calculation means 45a outputs, and a virtual body ] 
established beforehand, When the operation recognition means 45b has recognized the operation 
registered beforehand of "holding", it judges with the interaction, of "holding" having occurred 
and the value of the barycentric coordinates of the virtual body "is held" henceforth, which is 



memorized to the object memory measure 43 until an interaction is completed and which is held 
is coincided with the barycentric coordinates of cursor. Here, a value with actually larger cursor 
and an object than the distance which can contact in virtual space may be sufficient as said 
reference value defined beforehand. For example, it points to the operation which an operator 
will hold if the distance of a virtual body and cursor is below the reference value of said distance 
in the case of drawing 25 (arrangement of drawing 24) to the input means 1, and if the operation 
which the operation recognition means 45b holds is recognized, it will become possible to hold a 
virtual body and to move. 

[0096]When, as for the comprehensive judging means 45c, two or more virtual bodies exist 
below in the standard of said distance, It is aimed only at the object below a standard (for 
example, 90 degrees) with the angle of the line segment (wavy line) which ties cursor and a 
virtual body as shown in drawing 29 (b\ and the move direction (arrow) of the cursor which the 
movement vector calculating means 45d computes to make, The judgment of an interaction 
which took the move direction of cursor into consideration in an operators operation is attained 
(what is most located in the upper part among 3 objects by a diagram is chosen). 
[0097] About the migration length of cursor, when migration length is larger than the migration 
length standard established beforehand, an interaction is not caused. Thereby, when cursor is 
only being moved, it becomes possible not to cause the interaction which an operator does not 
mean. 

[0098] As shown in drawing 29 (c\ when two or more virtual bodies are meeting said standard, 
the comprehensive judging means 45c sets the virtual body near the position of the point of 
regard which the look input means 46 detects as the object "to hold" (the object of the left-hand 
side near the "+" seal which means the point of regard by a diagram is chosen). This becomes 
possible to choose an object easily using an operator's look. 

[0099]When the object which approached on the screen as shown in drawing 29 (d) exists, The 
comprehensive judging means 45c sets the virtual body which is in agreement with the shape of 
the cursor which the shape judging means 45e judged as the object "to hold" (since the interval 
of the finger of cursor is narrow by a diagram, it judges that a plane body is suitable as an object 
of the operation "to hold", and a plane body is chosen). This is enabled to choose the virtual body 
which an operator means with the shape of cursor, and operation becomes easy by matching the 
cursor shape of which it is easy to be reminded when an operator holds a virtual body. 
[0100]The comprehensive judging means 45c chooses preferentially the virtual body it was 
judged that was similar with the case where the learning means 45f has held the object in the 
past. Judgment near the operation which the operator performed in the past by this is reproduced, 
and it becomes possible to improve operativity. 

[0101]Thus, this example does not determine the existence of generating of the interaction of the 
cursor which an operator operates in virtual space, and the virtual body in virtual space only in 
the distance of cursor and a virtual body, It is possible by determining based on the example of 
the operation in an operator's operation, a look, or the past to raise operativity in the interface 
which performs an interaction with the virtual body using the cursor in virtual space. 
[0102]Although this example explained using the operation which holds a virtual body using 
cursor as an interaction, the treatment same about operation of the directions (pointing), the 
collision and friction and the blow to a virtual body, remote control, etc. otherwise is possible. 
The same effect is acquired, when virtual space is two-dimensional space, or also when a 
displaying means uses a three-dimensional three dimentional display display etc. It may realize 
using the software on a computer, using hardware as a realization means. 



[0103]This example the existence of generating of the interaction of the cursor which an operator 
operates in virtual space, and the virtual body in virtual space as mentioned above, Only the 
distance between components does not determine said cursor and said virtual body in virtual 
space, When a comprehensive judging means judges the existence of generating of an interaction 
by operation of the cursor which the distance and the operation recognition means during the 
representative point which a distance calculation means computes recognize, It becomes possible 
to cause an interaction also to the subject with which the distance in virtual space is not 
necessarily close, and it becomes possible to provide the good input/output interface of 
operativity. Since it is not necessary to calculate all the distance between components for said 
cursor and said virtual body in virtual space like the conventional contact diagnosis, 
computational complexity is reduced, and it becomes possible to attain improvement in the speed 
of processing. 
[0104] 

[Effect of the Invention]This invention can control easily the information displayed on the screen 
by the shape of a hand, and motion in the information displayed in the screen with sufficient 
operativity as mentioned above by carrying out a cursor display to a screen by making into 
special shape the feature of the shape of an operator's hand, or the shape of the hand which has 
recognized the motion again and has been recognized. 

[0105]Display on a screen by making the feature of the shape of a hand into special shape, and 
carry out a cursor display, and. By making automatically the judgment of an interaction which 
met an operator's intention in relation with display things other than the cursor display one by 
one, the interface whose operativity of operation of pointing to it or holding a display thing 
improved further is realizable. 



[Translation done.] 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing HT he outline view of the interface device by the 1st example of this invention 
[Drawing 2]T he detailed block diagram of the interface device by the 1st example of this 
invention 

[Drawing 31 (a) Figure showing the example of the shape of the hand judged with the - (c) said 
interface device 

[Drawing 4]T he figure showing the detailed example of the shape identification device of the 
interface device by the example . . 

[Drawing 51T he figure showing an example of the operation by the picture difference operation 
part of the example 

IDrawing 61 (a) Figure showing the example of the icon which the icon generation part of the - 
(d) said example generates 

[Drawing 71 (a) Outline view showing the example of the interface device by the - (c) said 
example of operation 

[Drawing 81T he outline view of the interface device by the 2nd example of this invention 
[Drawing 91T he detailed block diagram of the interface device by the 2nd example of this 



invention 

fDrawing lOlT he figure showing the example of the shape of the hand judged with the (a) and 
(b) said interface device 

FDrawing HIT he figure showing the detailed example of the operation recognition part of the 
interface device of the example 

[Drawing 121T he figure showing an example of the operation by the picture difference operation 
part of the example 

[Drawing 131 (a) Figure showing the example of the icon which the icon generation part of the - 
(d) said example generates 

FDrawing 141T he detailed block diagram of the interface device by the 3rd example of this 
invention 

FDrawing 151 The figure showing the operation recognition part of the interface device of the 3rd 
example by this invention 

[Drawing 161 (a) Figure showing the example of the icon displayed on a display screen with the 
interface device by the - (d) said example 

FDrawing 171 (a) Figure showing operation of the operation recognition part of the interface 
device by the - (f) said example 

FDrawing 181 (a) Figure showing operation of the operation recognition part of the interface 
device by the - (f) said example 

FDrawing 191T he figure showing operation of the operation recognition part of the interface 
device by the example 

FDrawing 201 The figure showing operation of the operation recognition part of the interface 
device by the example 

FDrawing 211 The block diagram showing the interface device in the 4th example of this 
invention 

FDrawing 221 (a) The figure showing the state where cursor opened in an example of the cursor 
used for the interface device of the example 

(b) The figure showing the closing ****** 

(c) The figure showing the state where cursor opened in other examples of the cursor used for the 
interface device of the example 

(d) The figure showing the closing ****** 

(e) The figure showing the state where cursor opened in the example of further others of the 
cursor used for the interface device of the example 

(f) The figure showing the closing ****** 

[Drawing 231 (a) The figure showing the form of an example of the virtual body used for the 
interface device of the example 

(b) The figure showing the form of other examples of the virtual body 

FDrawing 241 (a) The front view showing the cursor of virtual space, and arrangement of a virtual 
body 

(b) The side view showing the cursor of virtual space, and arrangement of a virtual body 
FDrawing 251 The figure showing the display example of the virtual space for describing the 
example 

FDrawing 261T he block diagram showing an example of the interface device of the example 
FDrawing 271 (a) The figure showing an example of the input device in the input means used for 
the interface device of the example 

(b) The figure showing other examples of the input device in the input means used for the 



interface device of the example 

(c) The figure showing the example of further others of the input device in the input means used 
for the interface device of the example 

rDrawing 281 (a) The figure showing an example of the picture which picturized the hand using 
the camera of the example 

(b) The figure showing an example which carried out binarization of the picture which picturized 
the hand using the camera of the example 

[Drawing 291 (a) The figure showing an example of the screen which the displaying means used 
for the interface device of this example displays 

(b) The figure showing the second example of the display screen 

(c) The figure showing the third example of the display screen 

(d) The figure showing the fourth example of the display screen 
rDrawing 301 The figure showing the conventional interface device 
[Description of Notations] 

1 Host computer 

2 Display 

3 CCD camera 

21 Frame memory 

22 Shape identification device 

23 Display control part 

24 Icon generation part 

25 Reference picture memory 

26 Reference image updating section 

3 1 Frame memory 

32 Operation recognition part 

33 Display control part 

34 Icon generation part 

41 Input means 

42 Cursor memory measure 

43 Object memory measure 

44 Displaying means 

45 Interaction judging means 
45a Distance calculation means 
45b Operation recognition means 
45c A comprehensive judging means 
45d movement vector calculating means 
45e Shape judging means 

45 f Learning means 

45 g Coordinate transformation means 

200 Icon 

201,202 Menu 

203 Icon 

204,205,206 Virtual switch 

221 Picture difference operation part 

222 Outline extraction part 

223 Shape identification part 



224 Position detector 

225 Shape change identification part 

231 Coordinate transformation part 

232 Coordinates pars inflexa 

261 Timer 

262 Picture updating section 

321 Outline extraction part 

322 Outline waveform operation part 
324 Coordinate table 

331 Coordinate transformation part 

332 Coordinates pars inflexa 

3231 Shape filter 1 

3232 Shape filter 2 

3251 Differentiator 

3252 Differentiator 

3261 Coordinates operation part 

3262 Shape judgment part 
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[0025] Zcr>X 0 ^SiaT'*4{R2BWfr£ISI 

cfcit4 temm b ggfflxwmwm, b co&m*?. $ 
f£t#S0-$- s^-y/K?)ift# tns^-y;K7)iiti 
touMtcto. t-vfrbvmgfflftcowm&bff) 

[0026] *30t<0JK«-Ctt. fiSfflHfcfcVvCiftfts 

0 tufms < fi - y /mj»#ai* 
( (Sffi^* 5 3 &7coi§£i2*?ffi ) raosgsio^fc i -5 

Srg£-r* l S*^JSS:^f^O*0tifi^(t. -^y^ 

[0027] 1 ommt. 

b, mi$k^mz$k7jkt i) ^ - y ;^(asfc ct im# 
*sogf iA^^ak. frie^-y^fia^ft^-ri. 
ft^^Sfc <t traffic - y rt^«tt*£Mrt-4 a - v 
/HEH^Sk. Btne*-y;Hi^i-c7)«^co{aas-^ 
•t h imgm ma x v im&mfovywst & am h m 
mmm. b . mt*> - y ;HB«#aKiBtrr 4 * - y 
;uofiSfcii^Kfc«^!fettseig#gcieei-4^ 

$&b<m<mmm%t&$em%*®.b , Me*- 
y/uw^4fcj«^<«wktig»r4iMH8»*s 
mieffiSitt»¥s*9mi-4E«fcMei!i^gai* 
?r ffl v vt line* - y >v b msMmi b o« 
Kmimttmitnz&mb&t>fS.?>zbzmib 

[0028] IffiMZ «t 0 . (KgSS^:^5V^•CS^#* { 

»fM-4 * -y ;ui &®mmftcowm#b <nm.^m 
cDm.^Mi. m.^m^mm^-v^bmiim 
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[0029] tfz. mi wmmz&^x . ®mm^& 
h sgsta^ *>f&bt& wxwtmi t=» ixm 

[0030] ttz. mi. m2<mmza^x. 
wiHzmh*-viv<nmiiiHtft®%.*%\&thft 

[ 0 0 3 1 ] S3«awycittvc. 

aaj* sama** & * - y /i^»fi# W5g#>£»i 

6SfS4tf)ffij£i: i/ctin. 

[00 32] 3S3. »4<0tt*tt$V^. flift^ 

mmzmj- 1 mmuzn Lxm-mm&m-m. 
[0033] tt:. mi~~m5wffimz}5^x. ti-v 

iv<m#. t mm <m#. t a<f*>si§! ttim^h^z 
& h %&zmKmm%mtfmm 

[0034] tit. mi-^mecom^za^x . t-v 

txmffism^mm&u wmmm&mm 

\iz^xm^^mwfn-Mm^m.^tmi 

[0035] ttz. mi-mKnmmz&wx. mwj 

mt&immmi&m&jmizxi. mm 
zm^tos&itcmmmuct&siz, wmxm 

[00 3 6] tit. m8<?>ffimz}i^X . SHAT^S 

im&thwmwm^mmmizttL. mm 
mmm±j&&iz*-v>w t &&u t^mfmm 

# StfffiSfWH £ &± § it I B 9 cr>m& 1 1 X t J: H . 

[0037] mi-m9ff)mmza^x. ffls^ 



OVffi&LtLXhJ:^. 

[0038] tti. miocoffimzts^x. x-vivt 

ft$Lbt&mmtn&&m&tt:i*mi*-v)WB 
ttxxmsmmnmm^mmziz^uz 
tiLwmtwffitttimtmGtz. m.mm.z 
mmmmmiZit&mi immtixhx^ 
[0039] ttz. mi-mi l^mmza^x. 

m&mnmhmi 2imt ix^xw 

[ 0 0 4 0 ] ttz. m 1 2<7)fflfi!t(;fcV^T. ffl£ffrH£ 

^-y^tw^frsa^ftitofiB 

[0041] minmm) mm. *fmzxu 
v?-7x4 xws&m i co$mm<w&* * u: *> o 

HCtJW, lli*Xh:jye. 3 .-;$\ 2li 
^fcffl^&r-fXTW . 3{iBfll5:»ffi-ri»CCD 
tityX'ht. CCDA/.y3\t. -T-fXrW 2C0« 
^JSTrti: H-*l«lK»«B*iai tTJ>0, 

-201.202ftW»E#»«*R«Lfc3K5Srff 57^f 3 

[ o o 4 2 ] 02ti. ^te^paayo ? ^ia$r^-r 

i>OT&&. CCD^^7 3* l <c>A7J$^B»Ji. 7 

: f#>imztit;AZ'&t%^imcDwm. mmwn 
Lxmfi>x\^h. mmmmit'&mzmixMmmm- 
zmtfx-tfi. 

[0043] B#.mXW®22te. y V~J±* t »J 

ikxbh<?>t>\ xm3(b)i ) z7jkti.o%z*m<r)miX' 

h h ^A'XIi-e 3 ( c ) X'h h tf)rt> Sr «W « . 

[ 0 0 4 4 ] H4li. ^Ki^'l^S22<7)P*ffl^ificO0ffi 
t*Lfct<0T. B«H«»a5221fclia8lfiiija5222i:}e 

U JtBfRt , S¥Bfift^ =E V fc«#3nfcWR 

{fa-if- 1 ws^^-^^grc # h mxh z.mt 



*. mmir&mtix&. mm* y vtimi-i 

[0045] flmttjMS223& fralBtttt5SE222tC J: 
36*, 0 3 ( a ) tC^-T i 3 & , * ^)*\ 

X03 (b ) \,zm X o %-*m<r>mKbh<r>fr*mfr 

[ 0 0 4 6 3 T-f 3y^gg24<i, ftMKS83i|g&223 lz£ 

¥<0^ttOlSWiM«*«-3|Ej|<0^KT$>*Uf , fl*.tf0 
6(a)(C5rfJ:3=fl«¥" 1 " <074 3y££j£U * 
fc3«&0»ttT*<itf . mtm 6 ( b ) CjjcfJ; 3 
2" ffiT-i ayZ&fcfh. TJnycoBtf. 

16(c) Cjjcf J: o £-*iBS0>7-f 3 y £ . XZ#£ 
flJWJtttMitf, 06(d) tSrf J: o %Z*fg§l(D74 

*. mta. tm^mzmrs^msT^ 3>^t 

*k k ttlWWJRk^AWB^tT**^-*-** 

[0047] J^<^C||«Lfc:4^WfcJ:*i*fW 
S:JaTt*-t. 07(a)fc^-f <fc?K. *58»«<0-fy 

-wnnmrnztht. t < a ±\,zm" r <r> 

TA^yftWrZlxhbb^z. l#S«0^-*-"C* 

* i k Ttt#«*)aS* 5EC«JBB-fi Z k iM^X'b 
I. ZZX\ m7(b)cr)X?lZttZ.*m<Qim)i<,ZtZ> 
b. r**7MJbfcler 2" W^ny^Jix 
*kkiC, 2m&<r>*-a.-X'hh*v\- r ?—7em. 

fsd&m&pztih. znw&x-mttt-i&ftam 
(ffiVi*mfi-&zbX'2%Bff)*=ijL-tfmmiiti. 

oizfi^m-itzb^x-^i,. t-jL-wmmzte^ 

PKb'ZmiXhXW H7(c)<0J:^CW»sftfe 
[0048] wtoi 9 fc*£»WfcJ:;fUf . g&UcB 

fSBQU miiq&mz&txzyex.- 
m<7)%8kiuw?z>zbff*jmx'$> f ), *-#-f* 

? £ k *^Itgt'?> I. . ¥«»gft£ll9HL*:tift* 
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*rfigk&*. 

[ o o 4 9 ] &*3, *3m0rai*ic*-A-»siiC]i 

fl*tf. £ fc Jfcb * i d Cffi t> TS£ 

bi>®%X'b&ZbliWotX'i>%\ l \ 

[00 50] m2<r>%jm) 08t±, *ft9ifcJ:ft4 

y 9 -y x 4 xmmnm 2 <nmm<wm*m l * t> o 

Tfc*. H8fc8*vc, mcoHJt0Jkffi«<3ttaB£W 
t*»»t:ttHt»ttTfllJ«l»*ji?t. HH^ Hi* 

Xh^yh'j.-^, 2tifUjHcffl^*r 4 XTVA , 3 

CCD;M5TJ>5. CCD*^53 

~ f-mw)^mkx-%hi. 3 >\iztt'>x^h. 

rW^f 2<7)^ffi±{ri(i, (KgX-f >yf-204.205,206,a 

203#«*T* S i ^ (c:^o tv^. 
[ 0 0 5 1 ] 09 li, *^Sfet0JB®«Pffl ; 2:^^7'Q 
7^05r^-ft<OTft§. CCD^^7 3^4>A^]$ix 

?tB«ji. y\y-^^)2uzmthiih. mm&x 

^'J25{c«i, ^«««SfutB«36*. ^BMktTft 
#8*1-0**. »W£g£t!K26t;i, ^-fv261kH« 
Mf?Si5262 k fc <k o LT $> 0 , ^^f V261TH^8 

B«S:«^B^ ; eU25tlE)!IL, S^Bii^Sfrr 
*. 

[0052] mmmumt. ? p-a^^ y 
£Bt£k . a^Bm^ ^ y izm-izixtm&b ton^tt 

»Db6*, WitfHl 0(a) £7Kt J: 3 =5r-*^^KT- 
* X0 1 0 ( b ) fc^-T i 3 0 ^tO^^T'J) 
*0*>S-WBtt*. 

[0053]|11«, »M2I8$22<?)p*ffl£^L*: t 

cot-, w&m%M&mbm$miii%222bBtf$zitm\ 

m25b&WMl&ffl24b£X^xnfSLlt:mZ*ltzi> 

<T)X'bh. ■ 

[0054] mmmmzzu^ mm^x o\,z7v- 

1** t U JtB« k . *^B®> * y 25tc«# 

8iTJtB«ktfOHSrSI*-r*. CimciO, WlZblX 
ttBJLfcv^**. 09i.{fi--f-O^o^kWJft^ 
^^^jrcl" * k HiStc, I8v v*Sft*B«>t"{t*ft 

aiT'^sfcftTj)*. wttfMimm%i$22izm&z 

5l«@SSt'«^U^^. 01 2t^-fJ:3tc, »B 
&*<?&<7)&ftb. 7l/-A^ty4><7)gfiBm+0# 

ogp^'tt $raaj-r * z t * ibtt-ci> o , ttv >x 

v**^cogp^«tSrfS#tc^t*|; **>lfCJ>* . fSI? 
}*tflSS222»i , B&£JgggB221 1 i o T MM Ltz&%?) 
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it. B*0>x -y v'S:Siaj-ri> £ 1 1 J: 0 . flUUzIM^ 

[oo55] m®tfmmzM*. ^mmmizx 

*mmxb&<r>t>K X01 0(b>fcij** <k?&g9 
[ 0 0 5 6 ] 74 ny£M%Mt. j£R£ftJR8'J»225 
Wf, fflt&mi 3(am*rf J:o&££|fcD74 3>£ 

<mmm-^m#X'hto,i, mi\mi 3(c) 

fi£U XH9**)»tt?JMitf, 0tttf013(d)(3S 
■fid*. «0»<o»«*«Lfc»tt«OT-f3V*4« 

[00573 «Un0]jSPg&23(i. 74 3y£lR«24fc J: o 
T£j£U:74 3 y^f -f 7.7W 2±T'0«^fiaS- 
«HPt*t>*>T. ffii0aft«231fcffliKt»232*»^ 
l> . ffig^mg|!231{i, «* LfcB«*>ffi«frfcT -f XT 

it. 3aiLfc«SfflW)S*fi«*RlKl-6. 
[ 0 0 5 8 ] BPfe . <ai$ifJgP224 t J: oTlfciiJ Lfci 

mte^Xir 2tC74 3y$r^-ri,i3(tT' 

[0059] a±<o«fc«jaLfc^aisffifcj:&a#M 

-^<Wc£ LT fc. ■?<f*7*M±(C*jSL 
iirfS. ftC. T4X7W 2±fc*SU:fflB0>«0B 

X4 •yf-204,205,206fiO±t#$:i)*^fct c J:-jT^ 

fc. *tf>fc«7.4 7 ^204, 205, 206 sWHffiSfU 
ayta-? llzmGZS-llZbtfTZ&hftX'Si 
6. 

[0060]^. ^aswcii. a«LfeB«+«)*ft 



[006 1] tLhOi 9 fc*iatWfcj:*Uf . JjW&U: 

ts mrnmiz^xvmitimmmr/xm 
m&togmitiftt&zkizx*). wmizj--?- 

#SO* { ^Bffi±{c^$its WiKasiffiJtfca 

x^<DAfl£*£^fci*"f . imzmmfflfrMfc 
■f>**I&k%hi>cDX'hh. • 
[006 2] (»3^Bt«)*W!Ht:J:4>fy^-7 

fcH8ti»iarc*9. w.2<r>%mmtntmms . 

HlO*ffl^TiftHHL, -£Ofi!iilfr.g>giS#<O*£014 

[0063] 01414. £3OHSfe»0pffi7Q.y?H 
**"f*>«"C**. CCD^^7 3^<oA^J§ii/2B^ 
{4 . 7 V-J±* * V 3 1 IzWt $>tlh . ISftfgiigS 3 2 
tt. 7V-A>*'J 3 1 W*tf 

10(b) c^-r j: 0 %m *) mmftx-ti h<»fr*vmt 

[ 0 0 6 4 ] 01 5ti. mmmm2<7)mM£*itzi> 
tfre&o. mi6-^2o$m^x*(DmMW)ft$mty 
+t>. mmizmzi it. wmiz&iEtmw 

Wfm&tfmrz* z, . hi 7 (am. mttibtimm 

[006 5] <J:(C|||Ii^^ia:gi5322{i. 01 7(a)fc 

B&zm^omte&szig&kLx. 0o^a<o^ri6j 

(R^K-III 0 ) fcKWttfcfc'if 0.019 tcS-f X 0 

s^Sr. mtk&sfrt>cr>$mnzm-m%.tixmiii 

U 01 7(b)t^-TJ:5=5r. 1 Z&flSWbbttL 

tzm>mi£$mthtmiz. mm uznm&ffli 

&±.<r)&&<r)Jm*7—7MzlX. £&t-7;P324 
izmtth . 3231&tX3232^-fJ^7 )V? 1 RWB 
«7^;^2»i, -e^-f^l 7(b)t*L^il}MB 

[0 06 6] J£t£7 -f /k? 1 1 J: 9 . 01 7(b){±. 0 
1 7<c)(cjtftJ: 3 (c. ^)g|5^IHIdi^SrRWLit 
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T, Hl7(d)CiirrJ:'3t. ISOlHlfl<0»tRttLfc 

->mft$ti. mmz. mn(e mvm i i ( f ) t 

Si^ffelB^K**, H 1 0 ( a) fcij?* J: 3 &Z*f£o» 
ttT**0>a\ X01 O<b)fcjp"f.i:3&«»)*0>#R 
TM<9*>£«rU B*(c£ll9l*tt3261M:. ffiK-x 

•c* § . mmmm26M , 017(e) tsr*- brum 

$ffl*¥<r). tttcfVixffl&tffttetl&'S.. lcl, 

1 c 2 £3<*\ *fl@ffllr-^324 t J: "5\ 02 OK 
[ 0 0 6 7 ] X. J£KflJ6g|53262tt s 111 7 ( f )<0#tt 

*<. mi? < >v? 2<r>&jM^ misM^-txo 

IHfl^V>#ttK»i:$:»K ftoT»#3s32620aj 
018(d)(c^-tJ:^{c. ^VkX«>%^J:3 

0, Hl0(b)(c^j:^iS:Hi3»O»«Tft*i:W5e 

;M-y vv-tmzm^hzttfx'th. 

[0068] HI A<r>T4n>£s&m\\±. 01 5«R> 

{fill 6(a)(C*tJ:?$r*EP07'f XS 1 ?* 
ft^KT*ittf> W*tfBl6<b)fcjj*-*-J:3$\ XEP 
OT-f 3y££j£-tl>. «^®|ffllgtB3{±, 7-f3y£j£ 
8B34K J: ->TM L£T4 3 V<?>t < tX'<n%. 

t>?4 zTujcot&jkmmwgmz'tT&K x. mm 

^Xt <X?V4\ZT4^V*%6tt%>hVtX'kh. Z 

[0069] y±<r>®t,zffi&ucimi&mco&ttm£a 



mcojmkizLxmfrtt. T<xrv4 2±£&j%u: 

7^3^203 «0^93*-V7W*<. #tf>i&#fc*tJSLfcffi 
feffittfUBRiU ^204,205,206 0>±£=t*W0f°tZ. 

yt-i-^ i izm%z$z.zz ktfx-z imx-bh. 

[0070] X, WoFthTA^s<hmthX\t. 01 
6(c), (d)(3j&fj:3lc« ^mtZmmTJ 3 

v\ attttfcfcL 016(c). (d)<7)J:33:B«£^tf> 
[007 1 ] -roi^fc:, #Stife#!l(i4 y*-7x4* 

tit Uz&m&co&Z&mk ix. * j f»c*m-*i& 

««7 4 iV9{zk^x%<mm&*m>^mm&z 
%®zti. &ot>z&&zmntztmmz. 

•y^-m^m'oizx^x&m&m^x'h') . -? 
m**i&t%&i><?>x'$,z>. 

[0072] ( «4 <r>3mm ) *«W«5»4 cOHifiMt: 
oV^TBIiB*«iLTK!!Bt*. 02 6ti-fy^-7x 

xna^^4^njfiM^7'o y ^0^^-rt><7)T-j) 

[0073] H2 6C&tYt. 4 lliAWg, 42Ji 

a, 4 5{sffl5^ffl¥iis#g. 4 5aawmsmm. 

4 5 bJiiifmi^S. 4 5c ttfg^HJgm 4 5 d 

ti^iK^ hA«aj*s. 4 5 e umimtt®. a 5 

[0074] 02 siza^xnftigtfAttm 1 \zn 
Lxw&'m\ mz&tt&mz&^x . ii-vtvm 
w&bA 2^-y;wtfo{sisortT'<o(ass-^-ti) 

>>-y;nete#a4 2#iEitt&*-v;^<5«^rt 



j vmimmtb £ 2 dstm^h tut 3 v&e&frt 
s. 

[0075] nmx^m $\tmm*±.\,zm 
-y;HEtt#S4 2 tfttt!eii#&4 3 tieittsm 

£tf>S*fc*r*v * - v iv b (Kg** t ofleBsnn 

^xmmt&m&m . m^9 vivn^m 

mm*%\$!>-th . J£Kfl£#&4 5 e [i#-y/K0$ 
ft k <EBft*0»«3ifi(B5:ff ffl t 

-y*fcfl^*^>«5ftffl*jB£ W:«te*>l!HE 

[0076] IS^ng¥R4 5 c titBftft£#«4 5 a 
^ffiflT^-y/l'kfo^ftfSkWSESIk, m%m 

S4 5 djftJflLaj'ti^-y/H^Blfcfrlfili: f*»SBik , 

*®ha7j#m$4 6 iwatM-* . mm%& 
ioffls^ffl t ^sms^ k Mie^ - v )v t m 

^tiEtTiuie^-y;i^ffisiKgft*^ft^ss 

gfl&iMM34 5 giifflsfBffpjjg 

#S4 5>WffiEftmiSjB£ Lfc*afcl0IBttt#«4 5 
aj5*KltWtcfflv^*-y^fcWk**fi0BSNl*i: 

[0077] 02 2(a), (b)U. >9-7 

(a>iij8*Hvvfctt», (b)\tmmttzmx'hh , 
022(c), (&)\x*wn<r>4y?-7±Axm.[zm 

fctf», (d)ti»fcmtfc«Wr**. 02 2(e), 

(f)(i*^^y^-7x-f^a(cfflv^i>^-y;u 

02 2fc£iVvt, (e)U^nvx^4QB. (f){i¥£BJ 
t 6. 023(a), (b)«*IBB<?5-f>^- 

((aHifcfrflc, (b)ti¥ffitt) . 

[00 78] &±<T)£ : Hzmf8.lt:4yi>-7x4zm 

srartJcj3^T02 2c7)j;3^-y^^iSL, fife 
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llzMlX'fio. 

[007 9] ZZX\ A3) l»Z\$il - V/WQMb £> \ ^t 
T, H2 7(a)Mc)fcjj^J:3fc*a*6Wi;M5 

JH<^&.rk#T#&. 02 7(a)liv?xx\ 
WffSNbt-&9'yn9V •y9%b'lz£^XJ)-VfriWi 
fct&. M2 7 (bmr~.99a-yx\ A«0#£g* 

u &<nmwft*T-9?u-7'<nmsmix'<r>m.* 
t-viKoimbmtizKt&zit. *-vn>&&m- 

1. 02 7(c)(ix3-f^x-f -/9X\ W<-«SMKJ 

k") £*M7T1§®U *tf»WKfe < kWM*K*i: 
[OO8O]028ti, *^^*^5-di«Uc*& 

x-tttm Ltifftx-b 028(a) <m&<7)%m%<?) 

M&ZZMfc Ltzh<r>tfm2&{h)X'bl. 028(b) 

iz&^x. mmmmz9mt&&i)B<m<7)m<?>%z t 
ifflffib**). %mm£focom.&imbWfefrt>{mb 

( ij - y ;K0^fj»^^ - y y WB*tf«aa|!Sr k' ) £ # 
[0081] *-VA«tfW4 2{iA^*K4 1 

*-y/i^^js<offle8siartT<?)«Bk»tt*E»r 

S. ft^k LX\ti}-V)V<mm>t^ (X0.Y0.Z0) 
fta^ikLTfflv^S. 

[ 0 0 8 2 ] LTti, «-y^«M»-« 

#ffltf)4" tffli^iaSlOlfflKr k'S-fflUTtiv^,,^ 
ktTJi02 2(a)<O^fiZf§cOiaBd, 02 2 
(b). (c)c0Jl^(±#tic0MI5c7)rtft^n ( n 
ff:en *vh§ < k MI5* { lft|^|.«®S:^) 
3BttflHB k 1 5 . =5:fc^k LT \tm<r>n^?^m 

<r>t&m£.®\*ix?>mm%b'zm^x t> «t v \ 
[oo83] mmm^m 3\mftmwLb%hm2 
3 &fttm^n<r>mm&<nmmmmbm& 

{SS%«c«0fi'C^ (*** : (Xl,Yl,Zl),¥SBtt : (X 

2. Y2.Z2) ) 

zvm&bLxm^z. %t3K%&bixi±. msm*. 
zffim-z&wco^im^m&cnimKb'zm^x & 

.fev-i. J^RkLT^i6S463t^tt5r*-f^*7^-^a 
Sr^Kk tTie^S ( C CiT'±^r*2: o = 1 , Vffi* 
Sr a = 2 k ) . %&B#b LTIlH^SS^k* 
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[0084] «**»4 4tt*-Vrt«tt#a4 2tm 

m^mnzm^ ^xmsm^ftg. i^m^a^ 

mbfM 2 d&tm^fh „ 02 9(a) t^^so^ 

[oo85] m.^m\^m 5\±*-v)vm.mi 

4 2tiBii^-i»*-y;^a^jg^ (xo.yo.zo) 
amm 3^m?>m.^si¥(7)m^mm (xi.yi.z 

1), (X2.Y2.Z2) fcc7)SE8i£ff-»-f&. 

[0086] mftim^m 5 b^mi^#>%®ite® 
zm^xmrntz . 02 2 ( a ) m - y /uo*£fci±z 

U 02 2(b), (c)o*-y;N0*^{C{i^f&OftJK 

[0087] h^*ai#a4 5 d(i^-y;i/<50 
m-brnm (xo.yo.zo) ^b^fflv^Tffi^sisirtfctJtt 

mwmcM (xo,Yo,zo)t fc-^fMoM'C^ (x 
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